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(54) System and method for identifying and selecting portions of information streams for a 
television system 



(57) A Profile Subsystem (which contains a profile 
of profile items in a database of information of interest to 
a viewer of a TV) will control selection of one or more of 
the segments of information if the segments themselves 
contain one or more profile items. In addition, an Infor- 
mation Processor will process the received information 
streams and segments to identify streams of interest to 
the viewers. In this case, the audio segments are proc- 
essed by any one or more of the following methods: 
using speech recognition to generate a searchable tran- 
script for key word searching, audio processing to deter- 
mine the type of audio including any one or more of the 
following: speech, music, noise, sounds, and speaker 
identification processing to determine the klentity of a 
particular speaker. The video segments are processed 
by any one or more of the following methods: using 
image processing to kJentify a searchat)le person, 
place, thing, action, or event. The data segments are 
processed by any one or more of the following methods: 
using text processing to generate a searchat)le hit list of 
one or more words. The Information Processor will gen- 
erate idemifiers from the processed segments and the 
identifiers will then be used by the Profile Subsystem to 
control the selection of information streams. 
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Description 

Retd of the Invention 

[0001] This invention relates to the area of digital tel- 
evisions. More specfficaJly. this invention relates to 
selecting and accessing different portions of an informa- 
tion stream from a digital television. 

Background of the Invention 

[0002] Analog television in the United States and else- 
where in the world is one of the most effective means of 
communication ard entertainment, yet It has undergone 
relatively little advancement in recent years in compari- 
son to other electronic technologies. Its principal capa- 
bility has been to distribute individual channels of 
moving images and sound to millions of viewers around 
the world. It has not. however, exploited the phenome- 
nal technical advances of the past twenty-five years nor 
the wealth of information now availal^e electronically. 
We are now on the verge, however, of enabling these 
advances with the introduction of digital television. 
[0003] Recently. webTV enabled the browsing of web 
p>ages and the display of them on a traditional television 
set. The television can be used for web browsing or pro- 
gram viewing, but cannot be used for both operations 
simultaneously. In both cases, the viewing is done in a 
communal manner, i.e. all viewers get the same infor- 
mation. Recently, some broadcasters have coordinated 
the simultaneous broadcasting of a particular program 
and the serving of associated information on the inter- 
net. Once again, this is done in a principally communal 
environment. 

[0004] Soon, digital television will enable the distribu- 
tion of auxiliary information with a video broadcast. With 
this technology, a video signal can be digitized, pack- 
etized and multiplexed with one or more audio signals 
and one or more data channels. The data channels can 
be closely related to the associated video or may be 
completely unrelated. These multiplexed signals are 
referred to as a program. Additionally, many programs 
can be multiplexed into a single Transport Stream. The 
resulting signsd can be distributed to television viewers 
through either a terrestrial broadcast, a CATV network 
or other network. Once received, the television can 
demultiplex the video, audio and auxiliary information 
and output it on the TV*s screen and/or speakers. 
[0005] Today, selection of the program content to be 
viewed on a television is accomplished by a channel 
selector It selects ttie VHF or UHF channel which the 
television will tune to and can often be controlled by a 
remote control device. In this capacity, the selector 
selects one video signal and one or more channels of 
audio. In addition, today s VCRs can be programmed to 
select a particular channel at a scheduled time in the 
future. Televisions do not typically have this feature. 
[0006] A television also uses many of the various 



types of wireless technologies oirrentty availafc)le (TV 
remote controls, broadcast signals, etc.). The broadcast 
signal is an example of a signal modulated to create a 
radio frequency signal which can be transmitted through 

5 the air. The television receives and demodulates this 
signal to create the original t)aseband signal. Television 
remote controls oftei modulate a signal on an infrared 
light to transmit information through the air In this case, 
the television recces and demodulates the signal to 

10 create the original baseband signal. Gat^e TV is an 
example of signals which have been modulated to a 
radio frequency and then transported over a coaxial 
cable. Once again, a television demodulates the signal 
to create the original baseband signed. In some cases. 

15 cable television networks modulate a laser or light emit- 
ting diode to transport the signal over a fibre optic cat>le. 
In ttiis case, a photo detector is used to enable the rec- 
reation of the original broadcast signal. 
[0007] Originally, a television was defined as a device 

20 capafc>le of receiving an analog broadcast signal over a 
VHF or UHF channel and selectively displaying one 
video signal togetiier witii its associated audio signal. 
Today, the signal received can come from a cable TV 
network, a terrestrial broadcast, a video tape recorder 

25 or camera, a computer network or other video source. 
As the television merges with other devices, such as 
personal computers, it will require an even broader def- 
inition. In this instance, a more appropriate definition is 
a device which receives a signal consisting of one or 

30 more video channels and one or more types of informa- 
tion strean^ including audio^ data, text and graphics 
and which outputs one or more of these streams. 
[0008] The signals output on a television today can 
include vkJea audio arxi data. The audio output is typi- 

35 cally amplified by an audio amplifier and tiien sent to a 
pair of headphones or one or more speakers. The video 
is most often displayed on a cathode ray tube, but it may 
also be displayed by a projection system or on a flat 
panel display. This latter technology includes ttie rela- 

40 tively new thin-film transistor displays. Text characters 
are also often overlayed on the video to display the time 
of day. selected channel, closed caption text or other 
information. Typically, one video stream, one text 
stream and multiple audio strecuns are output. 

45 [0009] Many televisions today contain processors of 
different types: video processors, digital signal proces- 
sors, etc. Televisions are beginning to merge with per- 
sonal computers and contain subsystems tiiat look like 
PCs. This may include a processor IC, static ram, 

50 dynamic ram. interrupt handlers and other peripheral 
devices normally associated with a computer. Thus, 
many of the applications which PC users have enjoyed 
will be available in the television system, including the 
capability to exchange messages and play computer 

55 games. 

[0010] Today, televisions do not assist in the notifica- 
tion to the viewer of other programs which may be of 
interest to the viewer. They also do not assist in the 
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selection of the program content to be viewed. In the 
near future, when auxiliary information will be distrib- 
uted with the broadcast, it will be even more important 
to identify and select information of interest to the 
viewer. The television system must be capable of assist- 5 
ing in this Identrfication. notification and selection proc- 
ess. 

[0011] It is an object of the present invention to pro- 
vide a technique which alleviates the above drawbacks. 

10 

Summary of the mvention 

[0012] According to the present invention we provide 
a profile system comprising: an information processor 
having one or more processors that processes a con- 75 
tent of one or more information segments to produce 
one or more identifiers associated with the content, the 
segments being part of an information stream broad- 
cast to a television output. 

[0013] A television companion device (TCD) is a ^ 
device comprising a communication transceiver capa- 
ble of receiving a signal from a television set. the signal 
having one or more types of information split from a 
broadcast signal received by the television set and an 
output that presents one or more segments of one or 25 
more of the types of information. This device enables 
the personal viewing of information selected specifically 
for the individual. This information can be different than 
and/or unrelated to that displayed on the television. The 
information displayed on each individual TCD may be 30 
video clips or video programs, audio clips or audio pro- 
grams, data, graphical images, still images, and/or ani- 
mations. The device may also comprise an input that 
sends one or more messages through the communica- 
tion transceiver to the television, thereby allowing the 35 
viewer to send requests for information to the television 
or send messages to other television companion 
devices. 

[0014] In this invention, if the automatic selection of 
information is desired, a Profile Subsystem (which con- 40 
tains a profile of profile items in a database of informa- 
tion of interest to the viewer) will control selection of one 
or more of the segments of information if the segments 
themselves contain one or more profile items. In addi- 
tion, an Information Processor will process the received 45 
information streams and segments to identify streams 
of interest to the viewers. In tills case, the audio seg- 
ments are processed by any one or more of the follow- 
ing methods: using speech recognition to generate a 
searchable transcript for key word searching, audio so 
processing to determine the type of audio including any 
one or more of the following: speech, music, noise, 
sounds, and speaker identification processing to deter- 
mine the identity of a particular speaker. The video seg- 
ments are processed by any one or more of the ss 
following metfiods: using image processing to identify a 
searchable person, place, thing, action, or event. The 
data segments are processed by any one or more of the 



following methods: using text processing to generate a 
searchable hit list of one or more words. The Informa- 
tion Processor will generate identifiers from the proc- 
essed segments and the identifiers will then be used by 
the Profile Subsystem to control the selection of infor- 
mation streams. This novel Information Processor can 
be used witti the TV and/or one or more TCDs. 

Brief Description of the Drawings 

[0015] The foregoing and other objects, aspects and 
advantages will be better understood from the following 
detailed description of preferred emtxxllments of the 
invention with reference to the drawings that are include 
the following: 

Rgure 1 Is a block diagram of an improved televi- 
sion system using a television companion device; 
Figure 2 is a block diagram of an improved televi- 
sion; 

Rgure 3 is a block diagram of a preferred television 
companion device; 

Rgure 4 is a block diagram of a Profile Subsystem; 
Rgure 5 is a flow chart of a selection process; and 
Rgure 6 is an example of a profile configured witti 
profile items. 

Detgiied Description of the invention 

[0016] With the advent of digital television, broadcast- 
ers will no longer be limited to broadcasting a single pro- 
gram of vkJeo and audio. They will soon have the ability 
to transmit multiple programs of audio, video, data and 
other information to the viewer. To capitalize on these 
new capEd^ilrties. a new television system is needed to 
facilitate the viewing of this information. 
[0017] Figure 1 1s a l^lock diagram of a novel television 
system 101 with various emtxxfiments of the television 
companion device (TCD) 120. In this system 101. the 
television 100 presents content to the general viewing 
audience 65 and oommunicates with a numt)er of televi- 
sion conrpanion devices 120 via a communication link 
(109. 121). In one preferred emkxxllment. the TCDs 120 
are handhekJ devices used to present incfividual. cus- 
tomized irrformation to viewers. In alternative embodi- 
ments. e.g. In a teleconferencing room or in a k>ar. the 
TCDs 120 can k>e mounted into a terminal. e.g. a table 
or desk, provided to each viewer 65. The goal here is to 
klentify. select and view infor ma tion streams of interest 
to the general viewing audience 65 (television) and for 
each individual 65 (holderAjser of a TCD). This will help 
the viewers (e.g. multiple numbers of 65) find and view 
the streams which have the highest entertainment or 
informational value to bdth the general audience 65s 
and the Individual viewer 65. 

[0018] The information streams 185 are audio (e.g.. 
English and/or Spanish commerrtary of a soccer game), 
video (e.g.. tiie moving images of the soccer game). 
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data (e.g. statistics on the players), graphic (e.g. graphs 
of the statistics), still image (e.g.. portraits of the play- 
ers) or animation streams (e.g. computer generated vkJ- 
eos diagramming and explaining a pl^ in the game) 
broadcast over a television network. In a preferred 
embodiment, these Information streams, or strearr^. 
typically 185. comprise a plurality of byte packets, typi- 
cally 188. as specified in the MPEG-2 specification. In a 
preferred embodiment, they are time multqplexed 
together to create a Program Stream, typically 1 86. One 
or more Program Streams 186 can then be multplexed 
together to create a Transport Stream (TS) 191 that is 
transmitted from the broadcaster. (A well known Pro- 
gram Identifier (PID) is used to identify each of the infor- 
mation streams 185 within the TS 191.) In Digital 
Television, the Transport Stream 1 91 is broadcast to the 
viewer 65 on a VHP or UHF channel. 
[001 9] As stated above, some of the important opera- 
tions are the identification, selection arKi presentation of 
the information streams 185 to the audience (65s). 
Although the television 100 must perform tiie selection 
for the general viewing audience 65s. the streams of 
interest 1 85 to tiie individual viewers 65 can be selected 
by the television 100 and/or tiie TCDs 120. As 
described below, the selection process can be executed 
automatically or can require manual intervention by the 
viewer 65. Once selected, the streams 185 for the gen- 
eral viewing audience 65s are output on the television 
100. The streams 185 selected for the indivklual view- 
ers 65s are transmitted by the television 100 to the 
TCDs 120 where they are output on a TCD output (125- 
127). e.g. an optional display device (126-127) and/or 
output on an optional audio device (125). 
[0020] The information streams 185 which are to be 
output on eittier the television 1 00 or TCD 1 20 must first 
be identified and selected. A preferred embodiment of 
this system includes a novel Selector (104 and 124 
below) which provides for the intelligent selection of 
streams 185 for a television 100 and/or TCDs 120. 
Since either the television 100 and/or the TCD 120 may 
p«form ttie selection, the selector (104/124) can be 
used in either or both devices (100/120). In one pre- 
tended emtxxiiment, the telension 100 and the TCD 120 
contain a selector (104/124). Thus, the television 100 
has the option of broadcasting all received streams 185 
to all the TCDs 120. a subset of tiie streams 185 to all 
the TCDs 120. a subset of ttie streams 185 to a sul>set 
of the TCDs 120. or only selected streams of interest 
185 to particular TCDs 120. In one preferred embodi- 
ment. e.g. during a particular time of day. only the TV 
Selector 104 selects the selected streams 185 and 
selects the sii^set of TCDs 120 to receive these 
selected streams. For instance, using this emtxxliment. 
a parent could linrtit information (stream 185) access by 
a child using the TCD 120. In an alternative emtxxli- 
ment. the TV 100 sends all streams 185 to all TCDs 
120. Here only the TCD 120 Selector 124 selects 
streams of interest 185 to display to each particular user 



65 on the respective TCD output (125-127). One appO- 
cation for this emtxxliment would be a public setting. 
e.g. a t>ar, with a large number of TCD users 65, each 
interested in specific statistics selected from a large 

5 number of statistical streams 185 atx>ut one or more 
games being viewed communally on the TV 100. In still 
another preferred emlxxlimerrt. lx>th the TV 100 and 
one or nrK>re of the TCDs 120 does tiie selection. In this 
case, the TV selects suteets of streams 185 to transmit 

10 to one or more sut^sets of TCDs 120. Then the TCDs 
120 will perform an additional selection of the incoming 
streeuns 185 to select tiie exact streams of interest for 
each particular viewer 65. In this embodiment, the TV 
100 selects basketball game streams 185 and baseball 

15 game streams 185 from all the sports streams 185 
received by the TV 100. Then each individual TCD 120 
user 65 selects which of the t>asketl>all/baset)all games 
and/or statistics he wishes to view. 
[0021 ] The communication link between the television 

20 100 and the TCD 120 can utilize a number of different 
technologies. In some implementations of the improved 
television system, the TV Transmitter 108 and TCD 
Receiver 128 (the communication link) can be radio fre- 
quency transmitters and receivers (witii respective TV 

25 antenna 109 and TCD antenna 121), fibre optic trans- 
mitters and receivers or coaxial drivers arKi receivers. In 
the former case, the signal will be transmitted through 
the air as in the case of the radio frequency broadcast 
described atxsve. In the latter cases, eitiier a fibre optic 

30 or coaxial cable will transport tiie C^omm Signal 140 
from the Television 100 to ttie'TCD 120. These technol- 
ogies are well known. 

[0022] Figure 2 is a block diagram of the novel TV 1 00 
and Rgure 3 is a block diagram of the Television Com- 

35 panion Device (TCD) 120 showing more detail. The TV 
100 contains a known Broadcast Antenna 101 used to 
receive a known terrestrial Broadcast Signal 1 50. In one 
preferred embodiment, the Broadcast Signal 150 is a 
digital signal which has been RF modulated using a 

40 known 8 Vestigial Side Band (VSB) modulator and ttien 
transmitted in the 6 MHz bandwidth of a traditional VHF 
or UHF channel. Typically, each VHF or UHF channel 
will contain one digital signal of up to approximately 20 
Mbps. In an altemative embodiment, tiiere is no 

45 antenna; the signal is received over a coaxial cat>le or 
other transmission technology. This digital signal com- 
prises different types of time-multiplexed information 
streams 185 including any one or more of the following: 
one or more video streams, one or nK>re audio streams 

50 (channels) and one or more data streams (channels). 
Each of these types of streams 185 can be further 
divided into segments (see Figure 4) where each seg- 
ment comprises a beginning, middle and end of any of 
the following: a video clip or program, an audio clip or 

55 program and a data segment. The streams and seg- 
ments are time multqplexed togetiier to create the digital 
signal 102A. These streams and segments are well 
known. 
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[0023] After being received by the Broadcast Antenna 

101 , the signal 150 is demodulated by the TV Receiver 

102. This will recreate 102A the original broadcast dig- 
ital signal oonrprising time multiplexed video streams, 
audio streams, and/or data streams 185. In high-end 
televisions, the TV Receiver 102 will be capable of 
demodulating multiple signals, thereby allowing the Tel- 
evision 100 to simultaneously select information 
streams 185 from difterent broadcast channels. The TV 
Demultqslexor 103 will demultiplex the video, audio and 
data streams 185 from the digital signal 102A into Indi- 
vidual segments of video, audio and/or data 109A. Inte- 
grated circuits are currentiy available which will 
multqplex and demultiplex the transport sta^eeuns 191 into 
a small nunr^er of individual information streams 185. 
They utilize the PID of each stream 185 to identify 
whether the stream 185 is audio, video or data. Some 
emt>odiments of the improved television system 1 00 will 
require a DemuKiplexor 103 which can output a large 
number of streams 185 simultaneously. This is done by 
cascading knowing Demultiplexors using known tech- 
niques. 

[0024] Once demultiplexed, the TV Selector 104. if 
used, will then select one or more stream(s) 185 to be 
optionally output on the TV 100 itself and to be sent to 
one or more TCDs 120 (through a novel TV Multiplexor 
113 (below), TV transmitter 108 and a TV communica- 
tion output, e.g.. Antenna 109. The TV Selector 104 is a 
specialized device which is capable of selecting one or 
more information streams 185 from a multitude of input 
information streams 185 received from the TV DemuKi- 
plexor 103. The TV Selector 104 is described in detail 
below. 

[0025] As the streams 185 are received, the optional 
TV Processor 112 will search the data within the 
streams 1 85 to locate important information such as the 
titie of a video segment, etc. This information can then 
be displayed by the TV Processor 1 12 on the TV Screen 
1 07 to inform viewers 65 of the existence of other infor- 
mation streams 185 which they may choose to select. 
The TV Processor 112 will generate menus which the 
viewer 65 can have displayed on either the TV Screen 
107 or the TCD Screen 127. If the menu is to be dis- 
played on tiie TCD Screen 127, the TV Processor 127 
will send the menu to the (TCD) Processor 134. This 
send operation is described later. As described below, 
the Menu Pointer 130 can be used to highlight an item 
in the menu and thereby select an information stream 
185 for output on either the TV 100 or TCD 120. 
[0026] The data, audio and/or video streams 185 
selected for the Television 100 will be sent to the TV 
Video Output 106 for display on the TV Screen 107 
(optional). In this embodiment, an MPEG-2 video 
decompression IC is used to decompress the video 
stream and create an analog video signal which can be 
easily displayed using known video output devices. The 
TV Screen 1 07 can be a CRT, a fiat panel display, a pro- 
jection display, a thin film transistor (TFT) panel or any 



8 

otiier type of graphical display The selected audio 
channel(s) will be optionally sent to the known TV Audio 
Output 105 to be output on TV speakers or head- 
phones. The TV Receiver 102, TV Demultiplexer 103. 
5 TV Audio Output 105, TV Video Output 106, and TV 
Screen 107 are well known. 

[0027] The TV Selector 1 04 itself is used in a new way 
to provide selected streams and/or segments 109A to 
the TV 100 and/or the TCDs 120. The Selector 104 will 

10 receive demultiplexed information streams 1 85 from tiie 
TV Demultiplexer 103 on separate well known data 
busses. As tiie streams 185 are received, the Selector 
104 will t)e instructed by the TV Processor 1 1 2 to select 
particular streams to be output on its separate output 

IS data trusses. In an attemative emtxxJiment, the streams 
185 are selected by a TV Profile Subsystem 110 
(below). Some of these output busses will send the 
selected information streams 109A to tiie TV Multi- 
jpHemr 1 13 for transmission to the TCDs 120. Otiier out- 

20 put busses will send the selected information streams 
109A to the known TV Audio Output 105 and/or tiie TV 
Video Output 106 for output on the TV 100. The method 
for actually selecting the streams 109A of interest is 
described below. 

25 [0028] The TV Multiplexor 113 will multiplex the 
selected information streams 1 09A from the TV Proces- 
sor 112 with information streams 185 from the. TV 
Selector 104 into Transport Streams 191 which-, can 
then be transmitted to the TCDs 120. The Transport 

30 Stream 191 is a well known format and can be found in 
tiie MPEG'2 Transport specification document. In this 
emtxxliment, the TV Multiplexor 113 novelly appends 
an Address Header field to the Transport Stream 191 
packets 188 to kientify the TCD120 which should 

35 receive the packets 188. In one preferred embodiment, 
each TCD is assigned an address that is used to deter- 
mine which of the packets 188 are to be received. A two 
byte header is attached to the beginning of each packet 
188. The first byte is tiie address of tiie TCD 120 and 

40 the second byte is a Linear Redundancy Check (LRC). 
The LRC is well known and is used to detect t)it errors 
which may occur in the address. The (TCD) Demulti- 
plexer 123 will only demultiplex and output the packet 
188 if the Address of the packet matches the address 

45 assigned to the TCD 1 20. In an alternative embodiment, 
the address is inserted in the user f iekis defined in the 
MPEG-2 Transport Specification and is utilized in the 
same way. In an alternative embodiment, the Demulti- 
plexer 123 demultiplexes all Information Streams 185 

so and the (TCD) Selector 124 compares the address of 
the Packet 188 to the address assigned to tiie TCD to 
determine if the stream 185 can be selected. 
[0029] In another alternative emt)odiment, the PID of 
each Information Stream 185 is used to determine 

55 which TCD(s) 120 should receive the Information 
Stream 185. If the sti-eam 185 was selected by the TV 
100 for a particular TCD 120, then the TV Processor 
112 will send a message to the (TCD) Processor 134 
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containing the PI D of the streanrs 185 which the TCD 

1 20 should receive. The Processor 1 34 will then instruct 
the Selector 1 24 to select the stream 1 85 which has that 
particular PID. If the stream 185 was selected by the 
TCD 1 20, then the Processor 134 will instruct the Selec- 5 
tor 124 to select the stream 185 which has that particu- 
lar PID. 

[0030] These streams/segments 1 09A which are des- 
tined for the TCDs 120 will be broadcast via the krrawn 
TV Transmitter 108 through the known TV output, e.g. 
Antenna 109. In some implementations of the improved 
television system 101 . the TV Transmitter 108 and TCD 
Receiver 128 will be Infrared transmitters and receivers, 
fibre optic transmitters and receivers or coaxial drivers 
and receivers. In the former case, the sigral will be 
transmitted through the air as in the case of the radio 
frequency broadcast descrit>ed above. In the latter 
cases, either a fibre optic or coaxial cable will transport 
the Communication (Comm) Signal 140 from the Televi- 
sion 100 to the TCD 120. In some implementations, bi- 
directional communications is needed t>etween the TV 
100 and the TCD 120. In thiscase, the TV 100 wfll com- 
prise a TV TCD Receiver 114 and the TCD 120 will 
comprise a Transmitter 135 utilizing any known commu- 
nication technology, e.g. one of the technologies out- 
lined above. 

[0031] In one embodiment, each stream or segment 
109A transmitted in this manner will contain an address 
290 which identifies the TCD(s) 120 which it is destined 
for. The address 290 can identify all of tiie TCDs 120 in 
the system, a sut)set of them or a particular TCD 120. If 
the transmission does not include an address 290, then 
the Television 100 will broadcast the information 
streams 185 to all the TCDs 120. In one preferred 
emtxxliment. the television 100 will not be able to select 
streams 185 for individual TCDs 120. The TCDs 120 will 
have to either output all the information streams 185 
transmitted by the television or it will have to use its own 
Selector 1 24 to select the streams 1 85 which will be out- 
put. In alternative embodiments, the PID of the informa- 
tion stream 185 can be used to identify streams 185 
which the TV 100 has selected for the TCD 1 20. 
[0032] Each TCD 120 will utilize its known Antenna 

121 to receive the Comm Signal 140 which is transmit- 
ted from the TV 100. The received signal 140 will be 
demodulated by the Receiver 1 28 to recreate the Trans- 
port Stream 191 transmitted from the TV 100. In some 
emkxxiiments where the signal is not modulated (e.g. 
some coax link technologies), the signal does not have 
to be demodulated tiy the Receiver 128. A (TCD) 
Demultiplexor 123 will demultiplex the received signal 
into indivkiual video, audio and/or data streams and/or 
segments 123 A. In some embodiments, the Demulti- 
plexor 1 23 is a novel demultiplexor which can utilize the 
Address Header described above to determine if the 
information stream 185 transmitted from the TV 100 
should be received tiy the TCD 1 20. It does this by com- 
paring its assigned Address to the Address in the 



Address Header which is appended to each packet 1 88 
in the Transport Stream 191 . 

[0033] Once demultiplexed, the (TCD) Selector 124 
will select which streams 185 will be output by the 
device. The selected data and video streams 124A will 
be sent to a (TCD) oulput, e.g. a Video Output 126 for 
display on a (TCD) Screen 127. The (TCD) Screen 127 
can be a CRT, a flat panel display, a thin film transistor 
(TFT) panel or any type of graphical display The 
selected audio channel(s) 124A also will be sent to a 
TCD output, e.g. the Audio Output 125 to be output on 
headphones or speakers and/or the Video Output 126 
after passing through a speech recognition system. As 
described above, these are known devices. Note that in 
alternative embodiments, one or wore of the TCDs 120 
would have only an audio oulput 125, e.g. if a first set of 
TCD 120 users 65 receives audio in a first language 
while a second set of TCD 120 users 65 watching the 
same TV 100 display receives audio in a second lan- 
guage. In another alternative embodiment, one or more 
of the TCDs 120 would only have a Video Oulput 126, 
e.g. where the TCD 120 users 65 are only interested in 
text information that supplements and/or augments the 
TV 100 video that is common to all the TCD 120 users 
65. 

[0034] The Selector 124 enables each individual user 
65 of a TCD 120 to select parts and subparts (seg- 
ments) of infomiation streams (transport stream, 
streams, and/or segment) 1 85 transmitted by the Televi- 
sion 100. Selection of the streams (segments) (109A 
and/or 124A) to be output by the TCD 120 is performed 
by the Selector 124 and/or the TV selector 104 which 
can be controlled in many alternative ways. For exam- 
ple, a Menu Pointer 130 can be used to select an item 
from a menu which appears on either the TV Screen 
107 or the TCD Screen 127. As described above, the 
TV Processor 1 12 will generate menus witii the names 
of various streams which the user 65 can choose from. 
Once selected, the TCD Processor 134 will instruct the 
Selector 1 24 to oulput the selected stream(s) 1 85. Alter- 
natively, a message can be sent from the TCD Proces- 
sor 134 via the Transmitter 128 to the TV 100 instructing 
the TV Selector 104 to select a p€u1icular stream(s) 
109A fertile TCD 120. 

[0035] The optional Keytx)ard 1 29 can also be used to 
identify a stream 1 85 to be output. The viewer 65 simply 
enters the name of the stream 185. As in the case of the 
Menu Pointer 130, tiie TCD 120 and/or ttie TV 100 will 
execute the selection 124 of the stream(s)/8egmerrt8. 
[0036] Finally, an optional Profile Sut)system 131 
and/or an optional TV Profile Sut>system 110 can be 
configured to use tiie selector (124 and/or 104) to select 
streams of interest 185 to the viewer 65. In this case, 
ttie S^ector 124 (TV Selector 104) will be automatically 
directed by the Profile Sut)system 131 (TV Proffle Sut>- 
system 1 1 0) to obtain these streams 185. 
(Q037] In one preferred embodiment tiie Selector 
(104. 124) is controlled by a Profile Subsystem 131 (TV 
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Prof9e Subsystem 110) which contains information of 
interest to the viewer 65. This infonnation can Include 
broad categories of interest (News, sports, weather), 
types of information (statistics, scores, finandai) or spe- 
cific information (person, place, things). The informa- 
tion, formally known as Profile Items (typically 605 
befow), is stored in a Profile Database 220. See Figure 
4 below. Each Profile Item 605 contains a category, a 
keywofd, and/or keyphrase which is compared to Identi- 
fiers (201 A 202A. and 203A in Figure 4 below) which 
identify the received streams 185. Identifiers (201 A. 
202A. and 203A) can be keywords or keyphrases within 
the information data streeuns 185 or can be generated 
by the information Processor (111 and/or 134) 
described below. 

[0038] Figure 6 shows an example of a Profile 600 in 
a Profile Sufc>system (110 and/or 134). In this endxxli- 
ment. the Profile 600 is a database of a plurality of 
records, typically 605. each containing a category field 
615. a Type field 620, and one or more fields with key- 
words and/or phrases 630. In this example, the general 
audience (television viewers) 65s are interested in any 
stream 185 of information related to Fires (entry TV 4). 
In the category of sports, the users 65s are interested in 
Information on any sports person named Pippen (entry 
TV 1) 630. any hockey scores (entry TV 2) 630 and any 
footiball scores 630 of the Atlanta team(s) (entry TV 3) 
630. The viewer of TCD 120 #1 is interested in world 
news 615 for Washington. DC 630 and Iraq 630 (entries 
TCD 1 -3 and TCD 1 -2) and any political news on Clinton 
630 (entry TCD 1-1). The viewer of TCD 120 #12 is 
interested in any events related to festivals 620 (entry 
TCD 1-4). any weather 615 forecast 620 for New York 
630 (entry TCD 12-1) and any information related to 
Finance 615 (entry TCD 12-2). In this embodiment, any 
combination of fields can be used to identify an area of 
interest. A Boolean AND operation is performed on the 
profile items 605 when more than one field (615, 620. 
630) has an entry 

[0039] As streams 1 85 are Received 1 28 and Demul- 
tiplexed 123, the Profile Suljsystem 131 and/or TV Pro- 
file Subsystem 1 10 will search for matches t>etween the 
received information streams 185 and the profile items 
605 in the Profile Database 220. The TV Profile Data- 
base 220 contains Profiles 600 for the TV 1 00 itself and. 
optionally, for one or more of the TCDs 120. The profiles 
600 for the TV 100 comprises profile Items 605 of inter- 
est to the general television viewing audience 65. The 
profiles 600 for tiie TCDs comprise profile items 605 of 
interest to each of the individual viewers 65 in the sys- 
tem. In addition, profile items 605 for a group of individ- 
ual viewers 65s can be described. The Profile Item 
Comparator 221 will perform the actual comparison as 
descrfoed below. 

[0040] In one prefened emtxxjiment. a match of all 
fields ( 615, 620. and 630) in the profile item 605 has to 
occur before there is a match. When a match is found, 
the selector (104 and/or 124) will be instructed to select 



the identified stream(s) or segment(s) (109A and/or 
124A) by using the PID to Identify the sti-eam 185. As 
described atx>ve, the TV 100 will transmit the selected 
stream(s) and/or segments 109A to the appropriate 

5 TCDs 120. In some embodiments, the TCD 120 will 
comprise a Profile Suk>system 131 which is the same as 
tiie TV Profile Subsystem 1 10 but which contains a pro- 
file 600 tor the TCD 120 only 
[0041] The optional TV Information Processor 111 will 

10 process the Incoming video, audio and/or data streams 
185 in search of information relevant to the viewers. For 
instance, Image processing can be used to identify a 
highlight of a footoall game. If a viewer were interested 
in such an event, the TV Profile Sut>system would be 

15 configured to Indicate this. When the TV Information 
Processor 1 1 1 Identified the event, it would output an 
Identifier (201 A, 202A. 203A) which will be compared by 
the Profile Item Comparator 221 to Profile Items 605 in 
the Profile Database 220. if a mateh is found, the Profile 

20 Item Comparator 221 will cause the TV Selector 104 to 
choose the appropriate stream(8) 185 and transmit 
them to the appropriate TCD 120. As described above, 
the TCD 120 would then oulput the sti'eam(s) 185 for 
the viewer 65. Alternatively, tiie Irrformation Processor 

25 132 located in the TCD 120 can process the Incoming 
stream(s) 185 and identify stream(s) of interest 185 to 
this particular viewer 65. In this case, the Selector 124 
would be instructed to select the appropriate stream(s) 
185. 

30 [0042] The TV Information Processor 111 and Infor- 
mation Processor 132 can utilize various technologies 
to process and Identify streams of interest 185 to the 
viewers 65. This is shown in Rgure 4. The Image Proc- 
essor 202 can be used to Identify a person, place, thing, 

35 action, or event which can be compared to the profile 
items 605 in the profiles 600. For example, the IBM 
Query by Image Content technology can identify a par- 
ticular color or texture in an image. Similarly, the Audio 
Processor 201 will use speech recognition to generate a 

40 searchat)le transcript for key word searching, audio 
processing to determine the type of audio including any 
one or more of the following: speech, music, noise and 
sounds and speaker kJentif ication processing to deter- 
mine the identity of a particular speaker. IBM s Via Voice 

45 product, for instance, can generate text from ttie spoten 
word. Finally, the Text Processor 203 can be used to 
reduce the incoming data information Into a searchable 
hit list of one or more words. The kientifiers (201A, 
202A, 203A) generated by these processors (201 , 202. 

50 203) are sent to the Profile Item Comparator 221 to be 
compared to items in the Profile Database 220. Each 
Identifier (201 A, 202A. 203 A) is compared to each entry 
in the profile 600. If all the entries in a record 605 are 
found in the set of Identifiers (201 A. 2Q2A. 203A) gener- 

55 ated for a particular Information Stream 185. then that 
Information Stream 185 is selected and sent to the TV 
100 or TCD 120 identified in the label of the entity 605. 
A match in this conrparison will result in an oulput on the 
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Selector Control 231 signal which will instruct the TV 
Selector 104 or Selector 124 to select the appropriate 
stream(s) 185. In an alternative emlxxliment. the Infor- 
mation Stream 185 will be selected if a sutjset of the 
Identifiers (201 A. 202A, 203A) for the Infbmriation 5 
Stream 185 matches any arbitrary subset of the fields ( 
615. 620. and 630) in the profUe item 605. The technol- 
ogies utilized in the TV Information Processor ill are 
all known. 

[0043] As an example of the improved television sys- ic 
tem 101, assume three television stations are broad- 
casting on different VHF channels: channels 3, 4 and 9. 
In this scenario, channel 3 is broadcasting a footiball 
game complete with highlights (NFL) and scores from 
other games in a data stream (SCORES). It is also 15 
transmitting news updates on a separate data channel 
(NEWSUP). Channel 4 is broadcasting a news program 
(NEWS), a news magazine show (NEWSMAG) and a 
documentary on US. Presidents (DOC). Channel 9 is 
broadcasting a movie in high<lefinition format (MOVIE). 20 
The profile 600 is configured as in the example above. 
There are two TCDs 120 receiving broadcasts from a 
television 100. Each TCD 120 has all of the optional 
components described above. Rgure 5 is a flowchart of 
the processes which occur in the improved television 2s 
system 101. 

[0044] Figure 5 is a flow chart of one preferred selec- 
tion process 500 that is used to performed by the sys- 
tem 101 . This process 500 is now explained by way of 
example. As the broadcasts are received, the TV 30 
Receiver 102 demodulates 501 the three broadcast sig- 
nals to recreate the original digital Transport Sti-eam sig- 
nals 191. The TV Demultiplexor 103 will then 
demultiplex 502 the signals to recreate the following 
information streams: NFL, SCORES, NEWSUP, NEWS. 35 
NEWSMAG, DOC. and MOVIE. All of tfiese streams 
185 will be sent to the Selector 104 for selection 503. 
Simultaneously, the TV Information Processor 111 is 
processing 512 each of the Incoming streams 185 and 
creating identifiers (201 A. 202A, 203A) which are used 40 
by the ProfDe Comparator 221 to determine 513 which 
streams 185 are to be output (105. 106, 125. and 126). 
These items are sent to the TV Profile Sut>system 110 
for use in the selection process. The TV Processor 112 
is also parsing 510 the data streams 185 to search for 45 
categories 615. types 620. and/or keywords and key- 
phrases 630 which are indicative of the content of each 
information stream 185. In addition, tiie TV Processor 
1 12 will also create menus 51 1 of tiie categories 615. 
types 620, and/or keywords and keyphrases 630 identi- so 
fied and will display tfiem on the Television 100 and 
TCDs 1 20 to allow the viewers to select streams of inter- 
est 109A and 124A 

[0045] In this example, the TV Information Processor 
111 has generated the Wentifier (201 A, 202A. 203A) ss 
FIRE from the newvs program (NEWS) after it has gen- 
erated the Identifier (201 A. 202A 203A) ATLANTA 
FOOTBALL from the NFL program (NFL). The TV Proc- 



essor 112 has also discovered the Identifier (201 A. 
202A. 203A) CLINTON in the documentary program 
(DOC) and the Identifiers (201 A. 202A. 203A) EVENTS 
FESTIVAL from the movie program (MOVIE). All of 
these Wentifiers (201 A. 202A. 203A) keywords are sent 
to the TV Profile Item Comparator 221 . 
[0046] The TV Profile Subsystem 1 1 0 is continuously 
conrrparing the received Identifiers (201 A, 202A. 203A) 
to the profile items 605 in the TV Profile Datat>ase 220. 
After finding a match between the kJentifier (201 A, 
202A. 203A) ATLANTA FOOTBALL and Profile 600 
entry TV 3 (620. 630), the TV Proffle Sutssystem 1 10 will 
instruct the TV Selector 104 to select 503 the footiball 
program (NFL) for the TV 100. L^er. when the FIRE 
Wentifier (201 A. 202A. 203A) is generated, the TV Proc- 
essor 1 12 will gen^^ate a menu 511 to be displayed on 
the TV Screen 107, This will alert the TV 100 viewers 
that a fire is being refenred to on the news program 
(NEWS) and will allow them to switch to this program. In 
addition, the TV Profile Subsystem 1 10 will insti^uct the 
TV Selector 104 to select 503 the documentary (DOC) 
for TCD 1 20 #1 and the movie (MOVIE) on channel 4 for 
TCD 120 #12. The streams selected for the TCDs 120 
are sent to the TV Multiplexor 1 16 where the addresses 
of each of the TCDs 120 are appended to the streams. 
The TV Multiplexor 1 16 will multiplex 504 these streams 
destined for the TCDs 120 together with the menus gen- 
erated 511 by the TV Processor 112. The Transport 
Stream 191 which is generated Is then transmitted 505 
to the TCDs 1 20 via the TV Transmitter 1 08 and the TV 
Antenna 109. 

[0047] As descrtoed previously, the TCDs 120 will 
receive 506 the Comm Signal 140 via the Antenna 121 
and will demodulate 506 the signal to create the original 
Transport Stream 191. The Demultiplexor 123 will 
demultiplex 507 ttie Transport Stream 140 into the indi- 
vidual information streams 185 and will compare the 
address of each stream with the address of the TCD 
120. When a match is found, the Demultiplexor 123 will 
fonward the stream to the Selector 124. The Seizors 
124 in TCDs 120 #1 and #12 will send the selected 508 
information stream(s) 185 to the appropriate device 
(125. 126) for output 509. Additionally, the Information 
Processor 132 and Processor 134 in the TCD 120 may 
also generate 51 5 Identifiers (201 A. 202A. 203A) on the 
information streams 185 received from the TV 100. Sim- 
ilar to the operation descrft)ed atxive, the Profie Sub- 
system 131 will compare Identifiers (201 A, 202A. 203 A) 
and will instruct 516 the Selector 124 to select 508 one 
or more of the information streams 185 for output 509 
on the TCD 120 if a sto^eam(s) 185 of interest to the 
viewer 65 have been found. 

[0048] Other capabilities enabled by the improved t^- 
evision system include the ability to exchange infbmia- 
tion between TCDs 120. In tfiis case, messages typed 
on the Keytx)ard 129 will be input to the Processor 134. 
The Processor 134 will fbmriat the message, embed the 
address of the TCD(s) 120 which should receive the 
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message and transmit the message via the Transmitter 
128. 

[0049] This will allow individuals 65 viewing a televi- 
sion broadcast to exchange messages with other view- 
ers 65 while viewing a television broadcast. 
[0050] In addition, the Processor 134 can run applica- 
tions such as games via the Processor 134. The broad- 
caster is also at)le to provide auxiliary information which 
can allow the viewer to play along with a game show 
being broadcast. This personalized version of the game 
would allow the viewer 65 to guess answers fc>efore the 
results were revealed on the show. 
[0051 ] The improved TV 1 00 will also provide access 
to the internet to obtain supplemental information. A 
viewer 65 can send an internet request by either typing 
the URL on the Keytx>and 129 or by using the Menu 
Pointer 130 to select a URL displayed on either the TV 
Screen 107 or the TCD Screen 127. The Processor 134 
will create a request message and transmit it to the TV 
100. The TV 100 will receive the message as described 
above and will instruct the Internet interface 115 to 
retrieve the web pages from the appropriate Internet 
site. Once obtained, the Internet Interface 1 15 will for- 
ward the pages to the TV Processor 1 12. The TV Proc- 
essor 1 1 2 will either transmit the pages to the TCD 1 20 
for output or will display them on the TV Screen 107. 
The processes for retrieving information from the Inter- 
net are well known. 

Clafnus 

1 . A profile system comprising: 

an information processor having one or more 
processors that processes a content of one or 
more information segments to produce one or 
more identifiers associated with the content, 
the segments being part of an information 
stream t>roadcast to a television output. 

2. A profile system, as in claim 1 , where the television 
output is any one or more of the following: a TV 
screen and a remote TV companion device output. 

3. A profile system, as in claim 1 . where one or more 
of the segments is selected to be outputted on the 
television output k>ased on one or more of the iderv 
tifiers. 

4. A profile system, as in daim 3. that further com- 
prises: 

a profile datat>ase having one or more entries, 
each entry having one or more fields describ- 
ing a criteria for selecting a segment; and 
a profile item comparator that compares one or 
more of the identifiers to one or more of the 
fields and selects a segment when the identifier 



matches one or more of the fields. 

5. A profile system, as in claim 4. where one or more 
of the segments selected is identified to a user to 

5 permit the user to select the segment. 

6. A profile system, as in any preceding claim, where 
the segments are any one or more of the following: 

10 an audio dip, a video segment, a still image, a 

stream of text, one or more graphical images, 
and one or more animated images. 

7. A profile system, as in claim 6, where one or more 
15 of the segments are audio segments that are proc- 
essed by audio processing to determine the type of 
audio including any one or more of the following: 
speech, music, noise and sounds and using 
speaker identification processing to determine the 

20 identity of a particular speaker. 

8. A profile system, as in claim 6, tiiat uses speech 
recognition to generate a searchable transcript for 
key word searching. 

25 

9. A profile system, as in claim 6. where one or more 
of the segments is are video segments that are 
processed by using image processing to identify a 
searchable person, place, thing, action, or event. 

30 

10. A profile system, as in claim 6. where one or more 
segments are data segments that are processed by 
using text processing to generate a searchable hit 
list of one or more words. 

35 

11. A profile system, as in any preceding claim, where 
tiie criteria includes any one or more of the follow- 
ing: a category, a type of information, a keyword, 
and a phrase. 

40 

12. A profile system, as in any preceding claim, where 
the field indudes one or more of the following IdrxJs 
of information: a proper name, a place, a thing, an 
activity, and a sport 

45 

13. A profile system, as in any preceding claim, further 
conr^rising a communications link 

14. A profile system, as in claim 13, where the commu- 
50 nication link is any one or more of the following: an 

infra red link, a radio link, a fibre optic link, and a 
coaxial link. 

15. A profile system, as in claim 13. that is located in a 
55 television and communicates through the communi- 
cations link with a remote television companion 
device. 
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16. A profile system, as rn claim 13. that is located in 
television companion device and communicates 
through the communications link with a remote tel- 
evision. 
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(54) System and method for identifying and selecting portions of information streams for a 
television system 



(57) A Profile Subsystem (which contains a profile 
of profile items in a database of information of interest 
to a viewer of a TV) will control selection of one or more 
of the segments of information if the segments them- 
selves contain one or more profile items. In addition, an 
Infomriation Processor will process the received infor- 
mation streams and segments to identify streams of in- 
terest to the viewers. In this case, the audio segments 
are processed by any one or more of the following meth- 
ods: using speech recognition to generate a searchable 
transcript for key word searching, audio processing to 
detemnine the type of audio including any one or more 
of the following: speech, music, noise, sounds, and 
speaker identification processing to detemoine the Iden- 
tity of a particular speaker. The video segments are 
processed by any one or more of the following methods: 
using image processing to identify a searchable person, 
place, thing, action, or event. The data segments are 
processed by any one or more of the following methods: 
using text processing to generate a searchable hit list of 
one or more words. The Information Processor will gen- 
erate identifiers from the processed segments and the 
identifiers will then be used by the Profile Subsystem to 
control the selection of infonmation streams. 
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